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Abstract 

In this research report, we review commonly held beliefs about test-optional policies and 
practices. Focusing solely on empirical evidence, we highlight research findings that directly 
address the stated intentions and actual outcomes of such practices. Throughout the paper, 
we raise concerns with test-optional policies as they pertain both to institutions as well as the 
students they serve. We conclude with the recommendation that colleges and universities 
employ holistic models of education readiness and success, supported by the notion that more 
information about students is better than less. 

Introduction 

In this report, empirical evidence that directly addresses the stated intentions and actual 
outcomes of test-optional practices is summarized. The five assertions reviewed in this 
report are: 

1. Test-optional policies increase the diversity of enrolled students 

2. Test-optional policies do not result in admitting less qualified students 

3. Test scores do not add any information above and beyond HSGPA 

4. Test scores are not predictive of college success beyond the first year of college 

5. Test scores are biased measures of student readiness for minority and underserved 
students 

The studies and empirical findings referenced within this document include a review of prior 
research findings found in peer-reviewed articles and technical reports, as well as original 
research. We conclude with the recommendation that stakeholders, including students, 
parents, and educators, use more information — not less — to evaluate one’s academic 
preparation level and make important educational decisions. 

Assertion l: Test-optional policies increase the 
diversity of enrolled students 

The stated goal of many institutions that have adopted test-optional admissions policies is 
to increase the diversity of students on their campus (Belasco, Rosinger, & Hearn, 2014; 
Espenshade, & Chung, 2011; Mattern, Shaw, & Kobrin, 2011). However, empirical evidence 
linking test-optional policies to increased diversity is lacking. In fact, a recent study of 
180 selective liberal arts colleges, of which 32 were test-optional, suggests that the diversity of 
the student body is largely unaffected (Belasco et al., 2014). On the other hand, one outcome 
that has been consistently impacted by test-optional policies is the number of applicants an 
institution receives. There is clear evidence that institutions that transition to a test-optional 
policy experience a substantial uptake in the number of students applying to their institution 
(Belasco et al., 2014; Epstein, 2009; Syverson, 2007). Results also indicated that test-optional 
institutions reported significantly higher test scores as compared to test-requiring colleges, and 
that the difference in test scores has increased over time. 

Given these data, concerns over the motives of institutions who adopt test-optional policies 
have been raised (Belasco et al., 2014; Diver, 2006; Epstein, 2009). Specifically, if more 
students apply, institutions can adopt a lower selection ratio providing greater leeway to be 
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more selective with the remaining predictors (e.g., high school grades, class rank, quality of 
courses completed). In addition to being able to admit a smaller percentage of applicants, 
the reported average test scores of enrolled students at test-optional institutions will likely be 
inflated since students with high admission test scores are more likely to submit their scores 
under a test-optional policy. Results from the Belasco et al. study (2014) suggest that this 
may be the case. A result, whether intentional or unintentional, may be an increase in the 
institutional ranking since student selectivity (test scores, high school rank, and selection ratio) 
accounts for 12.5% of the weight in determining where an institution falls on the U.S. News 
College Rankings. Perhaps more disconcerting is that students rely on reported institutional 
test score information to decide which schools to apply to and which schools to attend. If this 
information is faulty, we are doing a disservice to students by directing them towards colleges 
that may not be the best fit. 

Assertion 2: Test-optional policies do not result in 
admitting less qualified students 

Proponents of test-optional policies claim that such policies do not lead to admitting less 
qualified students. A study by Wainer (2011) explicitly tested this hypothesis by examining the 
performance of students from the Bowdoin College class of 1999. Matching Bowdoin data to 
College Board records, Wainer was able to obtain the test scores of Bowdoin students who 
did not submit scores. He found that non-submitters performed about 120 points lower on the 
SAT as compared to students who did submit their scores. Moreover, their first year grade 
point average (FYGPA) was substantially lower, as would have been predicted by their lower 
SAT scores. The average FYGPA was roughly 0.20 lower for non-submitters as compared to 
submitters. 

In a more recent study based on a multi-institutional sample, Hiss and Franks (2014) revisited 
this issue by examining college outcomes of students admitted under test-optional policies. 

The authors conclude that test-optional practices do not result in the admittance of less 
qualified students as there were few significant differences in the college performance of 
non-submitters versus submitters. However, the paper focuses primarily on the entire sample, 
which includes six public institutions (representing 58% of students in the total sample), which 
admit students based on high school rank and grades if the student has reached a certain 
grade threshold (e.g., HSGPAof 3.0 or higher) but still require admissions tests. We argue that 
the most relevant data to address the efficacy and validity of test-optional admissions policies 
would be the results from the 20 private colleges (representing over 37,000 students) admitted 
under what the authors define as “pure optional testing.” 

Focusing on the results for the students attending the private institutions, Hiss and Franks 
found that students submitting test scores had slightly higher grades in high school than non- 
submitters. In addition, test submitters outperformed non-submitters on freshmen grade-point- 
average (GPA) and cumulative GPA. Non-submitters were also less likely to declare a STEM 
major. Given that STEM fields tend to have more rigorous grading standards, the differences in 
grades between submitters and non-submitters would likely be larger if corrections for course 
difficulty were applied (Westrick, 2015). 
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In terms of predictive validity, the authors found that test scores accounted for additional 
variance beyond high school grade point average (HSGPA) for all institutions included in the 
study, and the effect was even more pronounced at private institutions using the “pure optional 
testing” model. However, the most compelling reason for using admissions tests is found 
when examining the validity of admissions decisions made based on non-submitters which 
increased from 0.16 with grades alone to 0.26 when admission test scores were incorporated 
for the private colleges. When examining results for the “pure optional testing” model, the 
evidence clearly indicates that admissions tests add important incremental validity in predicting 
college success beyond HSGPA. These results are likely an underestimate of the efficacy 
of admissions testing because it didn’t control for different course taking patterns between 
submitters and non-submitters. Research has shown STEM courses are more stringently 
graded than non-STEM course grades (Westrick, 2015). Similarly, other differences in the 
academic preparation and background of test score submitters and non-submitters were not 
addressed. 

Assertion 3: Test scores do not add any information 
above and beyond HSGPA 

Critics of standardized tests often point to the small additional variance accounted for in 
college outcomes by test scores after taking into account HSGPA as evidence that test scores 
do not provide meaningful information beyond high school grades (Mattern, Kobrin, Patterson, 
Shaw, & Camara, 2009). However, the same argument applies when evaluating the additional 
variance accounted for by HSGPA over test scores. A recent meta-analysis of four-year 
colleges indicates that both HSGPA and ACT® test scores are strong predictors of FYGPA 
(Westrick, Le, Robbins, Radunzel, & Schmidt, 2015); albeit the relationship between HSGPA 
and FYGPA was stronger (r = 0.58) as compared to that of test scores and FYGPA (r = 0.51 J. 1 
With that in mind, HSGPA will account for a larger percentage of variance beyond test scores 
than vice versa when predicting FYGPA; however, that does not negate the importance of test 
scores. 

As an alternative method for illustrating the added value of test scores, researchers have 
begun to show graphically the differences in expected outcomes at various HSGPA and test 
scores values (Bridgeman, Pollack, & Burton, 2004; Sawyer, 2010). For example, as shown in 
Figure 1 , the probability of earning a B or higher in the first year of college varies dramatically 
among students with the same HSGPA but different ACT Composite scores (Sawyer, 2010). 
Among students with a 4.0 HSGPA, students with an ACT Composite score of ten have less 
than a 30% probability of earning a B or higher as compared to over a 95% probability for 
students with an ACT Composite score of 30. Even for less extreme cases, the results illustrate 
that test scores meaningfully discriminate among students with the same HSGPA. Therefore, 
institutions that ignore test score information would in effect consider all applicants with the 
same HSGPA as having the same likelihood of being successful once on campus; however, 
the results indicate that this is not the case. 


Correlations were corrected for range restriction due to selection based on ACT score, HSGPA, and socioeconomic 
status. 
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HSGPA 

Figure 1 . Probability of 3.00 or Higher FYGPA, Given HSGPA and ACT Composite 
Score. Reproduced from Richard Sawyer (2010). Usefulness of High School Average 
and ACT Scores in Making College Admission Decisions. ACT Research Report 
Series 2010-2. Iowa City: IA. 

Test-optional policies can foster scenarios where applicants with lower HSGPAs but higher test 
scores are being denied admissions over applicants with higher HSGPAs but lower test scores, 
even when the denied applicant is more likely to be successful. For example, a student with 
3.2 HSGPA and an ACT Composite score of 25 has roughly .63 probability of earning a B or 
higher. On the other hand, a student with a 3.5 HSGPA and an ACT Composite score of 20 has 
a .59 probability. These examples illustrate the additional information gleaned from taking 
both students’ HSGPA and their test score in consideration as more information leads to more 
informed decisions. Moreover, having multiple pieces of information allows institutions to more 
accurately identify students on their campus who are at-risk academically and then provide 
resources appropriately. Ignoring test score information may result in the failure to identify 
at-risk students; thus, their likelihood of succeeding and staying in college may be diminished 
because they may not receive additional support. 

Assertion 4: Test scores are not predictive of college 
success beyond the first year of college 

Another misconception is that test scores are not predictive of more important long-term 
outcomes such as cumulative grades or college graduation. Contrary to popular belief, 
empirical data indicates that test scores are predictive of more distal outcomes (Radunzel & 
Mattern, 2015; Radunzel & Noble, 2012). In fact, data indicate that students who graduate 
high school ready for college as indicated by their ACT test scores are significantly more 
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likely to enroll in college and graduate within six years (Figure 2; Radunzel & Mattern, 2015). 
For example, among students who met all four ACT College Readiness Benchmarks, six out 
of ten are predicted to immediately enroll in college and complete a degree within six years. 
Alternatively, of students who did not meet any ACT College Readiness Benchmarks, only two 
out often are expected to immediately enroll in college and complete a degree within six years. 
In fact, only half of all students who did not meet any benchmarks are expected to enroll in 
college immediately following high school graduation. One potential criticism of these findings 
is that they do not take FISGPA into account. Since HSGPAand test scores are correlated, 
readers may be wondering if test scores still provide added value in terms of predicting 
long-term outcomes once HSGPA has been taken into account. 


Number of ACT 

College Readiness Number of projected college enrollees 

Benchmarks met and graduates for every 10 students 

~ ttttt 

tttttttt 

tttttttt 

ttttttttt 


Will not enroll in college immediately 

* Immediately enroll in college and will 
not complete degree within 6 years 

t Immediately enroll in college and 
complete degree within 6 years 


Figure 2. College Enrollment and Graduation Rates by Number of ACT College 
Readiness Benchmarks. Reproduced from Radunzel, J., & Mattern, K. (2015). 
Providing context for college readiness measures: College enrollment and graduation 
projections for the 2015 ACT-tested high school graduating class. Iowa City, IA: 

ACT, Inc. 


Turning our attention to the research evidence on the joint influence of test scores and HSGPA 
on college success beyond the first year of college, research findings consistently point to 
the fact that ACT test scores do provide incremental validity above and beyond HSGPA in the 
prediction of long-term college outcomes such as cumulative grades and college graduation 
(Radunzel & Noble, 2012). Figure 3 illustrates this point using six-year cumulative GPAas 
the outcome of interest. Specifically, among students with the same HSGPA, students with 
higher ACT Composite scores are more likely to earn a six-year cumulative GPAof 3.00 or 
higher than students with lower ACT Composite scores. For example, among students with 
a 3.0 HSGPA, students with an ACT Composite score of 20 have roughly a 0.40 probability 
of earning a cumulative GPAof 3.00 or higher as compared to a nearly 0.70 probability for 
students with an ACT Composite score of 35. The differences in probability of success by 
ACT score become larger at higher levels of HSGPA — the part of the scale where admission 
decisions are often made and the requirement to submit test scores is often waived. These 
results suggest that test scores are even more informative for students with higher HSGPAs, 
which is in direct contradiction to test-optional policies. 
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Figure 3. Six-year Cumulative GPA of 3.00 or Higher by HSGPA and ACT Composite 
score. Reproduced from Radunzel, J., & Noble. J. (2012). Predicting Long-Term 
College Success through Degree Completion Using ACT® Composite Score, ACT 
Benchmarks, and High School Grade Point Average. (ACT Research Report No. 
2012-5). Iowa City, IA: ACT, Inc. 

We find a similar pattern of results when examining six-year bachelor’s degree completion 
rates, though the differences are not as large as those seen for cumulative GPA (Radunzel 
& Noble, 2012). Among students with a 3.0 HSGPA, students with an ACT Composite score 
of 20 have a 0.34 probability of earning of a bachelor’s degree in six years as compared to 
a 0.41 probability for students with an ACT Composite score of 30 (Figure 4). These findings 
underscore the importance of considering multiple pieces of information to better understand 
students’ levels of readiness for college and their likelihood of being successful in college, 
starting from the first day through college graduation. 
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Figure 4. Six-year Bachelor’s Degree Completion Rates by HSGPA and ACT 
Composite score. Reproduced from Radunzel, J., & Noble. J. (2012). Predicting 
Long-Term College Success through Degree Completion Using ACT® Composite 
Score, ACT Benchmarks, and High School Grade Point Average. (ACT Research 
Report No. 2012-5). Iowa City, IA: ACT, Inc. 

Discrepant Performance 

Another way to think about the added value of test scores beyond HSGPA applies to cases 
where students perform significantly higher on one measure as compared to the other 
(Mattern, Shaw, & Kobrin, 2011). National data indicate that the nearly three-quarters of 
students perform similarly on the ACT as compared to their HSGPA; however, a fair number 
of students have a significantly higher HSGPA as compared to their ACT test scores or vice- 
versa. To identify discrepant performance, the difference between students’ standardized 
HSGPA and ACT Composite score was computed. Students who scored one standard 
deviation or higher on HSGPA as compared to their ACT Composite score were categorized 
as Higher HSGPA whereas students who scored one standard deviation or higher on ACT 
as compared to their HSGPA were categorized as Higher ACT. Students who scored within 
one standard deviation on the two measures were categorized as Non-Discrepant. Based on 
the 2015 ACT-tested graduating cohort who had both a valid ACT score and HSGPA, which 
constituted over 1.6 million students, we found that 13% (-200,000) had significantly higher 
ACT Composite scores, 74% (1.2 million) had non-discrepant performance, and another 13% 
(-200,000) had significantly higher HSGPA. These percentages are driven by the one standard 
deviation rule for determining discrepancy — a much larger percentage of discrepant students 
would be observed, for example, if discrepancy was based on a half standard deviation rule. 
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Examination of the student characteristics of the three discrepancy groups reveals significant 
differences in gender, parental education level, household income, and race/ethnicity. We also 
find that Higher ACT students attend high schools that differ in meaningful ways as compared 
to the high schools that Higher HSGPA students attend. In particular, differences in percentage 
of students who are college-bound at their school, the percentage of the student body 
qualifying for free or reduced lunch (FRL), and school location emerge. As shown in Table 1, 
females, students from lower socio-economic status (SES) backgrounds, and underserved 
minority students comprise a larger percentage of the Higher HSGPA group as compared to 
the other two groups. In terms of school characteristics, students with higher HSGPA are more 
likely to attend high schools in rural areas, serving a larger percentage of students eligible for 
free or reduced lunch, and having a smaller percentage of students who are college-bound. 
These results suggest that HSGPA and ACT scores are not redundant measures. Instead, 
discrepancies in the two measures vary systematically by student and school characteristics. 
The results by socio-demographic characteristics highlight the disparities in educational 
opportunities that currently exist is the US where many underserved minority students attend 
high-poverty elementary and middle schools (Ross, Kena, Rathbun, et al., 2012). 
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Table 1 . Student Characteristic of 2015 ACT-tested Students by Discrepant Category 


Higher Non- Higher 

Student Characteristics ACT discrepant HSGPA 


Number of Students 


212,476 

1,211,589 

208,520 

Academic 

ACT Composite 

24.5 

21.5 

16.7 

Preparation (mean) 

HSGPA 

2.66 

3.27 

3.60 


Female 

37.7% 

55.3% 

63.5% 

Gender 

Male 

61.7% 

44.2% 

35.8% 


Missing 

0.6% 

0.5% 

0.6% 


No College 

17.1% 

18.6% 

28.0% 


Some College 

21.4% 

24.8% 

27.6% 

Parental Education 
Level 

Bachelor’s Degree 

23.6% 

25.8% 

19.8% 


Graduate Degree 

25.1% 

19.8% 

11.0% 


Missing 

12.8% 

11.0% 

13.6% 


Less than $36,000 

21.0% 

23.6% 

35.3% 


$36,000 to $60,000 

15.0% 

16.7% 

17.9% 

Income 

$60,000 to 100,000 

16.9% 

19.0% 

15.1% 


Over $100,000 

22.5% 

19.7% 

10.4% 


Missing 

24.6% 

20.9% 

21.3% 


Black/African American 

9.3% 

12.5% 

19.9% 


American Indian/Alaska Native 

0.6% 

0.7% 

1 .2% 


White 

59.4% 

58.5% 

43.1% 

Racial/Ethnic 

Hispanic/Latino 

12.7% 

14.7% 

22.3% 

Background 

Asian 

6.3% 

4.4% 

5.0% 


Native Hawaiian/Other Pacific Islander 

0.3% 

0.3% 

0.4% 


Two or more races 

4.7% 

4.0% 

3.5% 


Prefer not to respond 

6.7% 

4.9% 

4.7% 


<50% 

8.0% 

9.6% 

12.5% 


50-70% 

26.9% 

31.9% 

39.2% 

School College- 
bound Rate 

70-80% 

18.6% 

20.3% 

20.7% 


80-90% 

22.5% 

20.9% 

17.0% 


>90% 

24.0% 

17.4% 

10.6% 


0-20% 

30.0% 

20.8% 

7.6% 


20-40% 

31.8% 

30.7% 

22.0% 

School Poverty 
(FRL%)* 

40-60% 

23.0% 

26.8% 

29.8% 


60-80% 

10.4% 

14.4% 

23.7% 


80-100% 

4.9% 

7.4% 

16.9% 


Rural 

16.8% 

22.7% 

26.2% 


Town 

10.1% 

13.4% 

15.0% 

School Locale* 

Suburban 

45.7% 

40.0% 

31.5% 


City 

27.4% 

24.0% 

27.2% 


* only available for students attending a public high school. 
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The results presented thus far clearly indicate that students with a particular HSGPA vary 
considerably in terms of their ACT Composite scores; likewise, students with a particular ACT 
Composite score vary considerably in terms of their HSGPA. A graphical representation of this 
variability is displayed in Figure 5 where larger boxes indicate a larger number of students 
who have a HSGPA and ACT Composite score combination in the specified ranges (Camara 
& Moore, 2016). For example, an ACT Composite score of 22 to 24 and a HSGPA of 3.75 or 
higher is a common combination as indicated by the larger box. Interestingly, among students 
with low HSGPAs (<2.50), there is less variability in the distribution of test scores. That is, both 
measures suggest the student is not academically prepared. On the other hand, the variance 
in test scores among students with a high HSGPA (>3.75) is much larger with large blue boxes 
across the entire ACT score range except for the bottom two ACT test score range categories. 
For many of these students, the measures are providing conflicting information. Such findings 
beg the question, which measure is the true indication of the student’s level of academic 
preparation for students with discrepant performance? 

ACT Composite 


HSGPA 

1-12 

13-15 

16-18 

19-21 22-24 25-27 
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Figure 5. Distributions of High School GPAs and ACT Composite Scores. 
Reproduced from Camara, W. & Moore, J (2016). Research you can apply to 
practice. “Really”! Presented at the annual Enrollment Planners Conference, 
Chicago, IL. 

The extent to which these measures — individually and in combination — accurately predict 
future outcomes for discrepant students speaks to this question. To address this question, 
three different regression models were estimated based on a sample of ACT-tested students 
who also had FYGPA data available 2 : 1.) a model that relied solely on HSGPA to estimate 
one’s predicted FYGPA, 2.) a model that relied solely on ACT Composite Score to estimate 
one’s predicted FYGPA, and 3.) a model that included both HSGPA and ACT Composite to 

2 The sample included nearly 400,000 students, representing 289 post-secondary institutions. 
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estimate one’s predicted FYGPA. For each regression model, a student’s predicted FYGPA 
was compared to their earned FYGPA. A residual or error term was computed for each student: 

Residual = FYGPA —FYGPA , , ,, 

earned predicted’ 

where positive residuals indicate that the student earned higher grades in the first year than 
what the model predicted. In other words, their performance was under-predicted. Negative 
residuals indicate that the student earned lower grades in the first year than what the model 
predicted; their performance was over-predicted. Given the methodology used to fit the data 
(i.e., ordinary least squares), mathematically, the average residual across all students must 
equal zero. Flowever, the average residual for subgroups in the sample may diverge from zero. 
We are interested in examining the degree to which the three models results in over- or under- 
prediction for the three discrepant groups. The results are summarized in Table 2. 

Table 2. Mean (SD) Residual by Discrepant Group 

Higher HSGPA Non-discrepant Higher ACT 

Predictor(s) (N = 42,114) (N = 312,159) (N = 38,362) 

HSGPA -0.31 (0.88) 0.02 (0.83) 0.20 (0.95) 

ACT 0.37 (0.87) 0.02 (0.85) -0.60 (0.98) 

HSGPA & ACT -0.05 (0.86) 0.01 (0.82) -0.05(0.96) 

Note. All mean residuals are significantly different than zero at p <.01. The effect sizes for the Higher HSGPA 
and Higher ACT group for the HSGPA and ACT models are small to medium. Negligible effect sizes are 
observed for the non-discrepant group for all three models and for the discrepant groups for the HSGPA 
& ACT model. 


So which measure is a better indicator of students’ academic preparation or their likelihood of 
success when the measures provide conflicting information? Table 2 indicates that prediction 
models that use only one piece of information result in more prediction error for students 
with discrepant performance. For the model that included only HSGPA to predict FYGPA, 
students with discrepantly higher HSGPAs earned lower grades in college than what the 
model predicted (mean residual = -0.31). For example, if this group of students was predicted 
to earn a FYGPA of 3.31 , in reality, they earned a 3.00, on average. Conversely, the HSGPA 
only model under-predicted FYGPA for students who have discrepantly higher ACT scores 
(mean residual = 0.20). That is, these students earned higher grades in college than what the 
model predicted. Students with non-discrepant performance were accurately predicted (mean 
residual = 0.02). These findings suggest that test-optional policies or only relying on HSGPA 
to make admission decisions unfairly advantages students with discrepantly higher HSGPAs 
while unfairly disadvantages students who have discrepantly higher ACT scores. 

The pattern of results reverses when focusing on the ACT only model. Students with 
discrepantly higher HSGPAs earned higher grades in college than what the model predicted 
(mean residual = 0.37). Conversely, the ACT only model over-predicted FYGPA for students 
who have discrepantly higher ACT scores (mean residual = -0.60). Students with non- 
discrepant performance were accurately predicted (mean residual = 0.02). Though we are not 
aware of any HSGPA-optional colleges, the same warning given for the HSGPA-only model 
applies here. Institutions that use admission test scores as the sole criteria for entry into an 
honors program, for example, would likely risk over-prediction of some students. That is, 
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relying only on test scores to make important decisions may unfairly disadvantage students 
with discrepantly higher HSGPAs while unfairly advantage students who have discrepantly 
higher ACT scores. 

Finally, the model that included both ACT scores and HSGPA resulted in the least amount of 
differential prediction for each of the discrepancy groups. The mean residual was -0.05 for 
both the discrepantly higher HSGPA group and the discrepantly higher ACT group. The mean 
residual for the non-discrepant group was 0.01 . The results suggest that one measure is not 
better — ora more accurate indicator — for students with discrepant performance. Rather, taking 
into consideration both pieces of information results in more accurate predictions and neither 
piece of information should be disregarded or ignored. This is especially true for discrepantly 
higher HSGPA students, who are the ones most likely to take advantage of and benefit from 
test-optional policies. In addition, admissions professionals rarely make decisions through 
purely mechanistic means; they base decisions on a comprehensive review process which 
allows them to give greater or lesser weight to evidence which is discrepant or inconsistent 
with a students’ entire academic record, thus reducing the rationale for not considering any 
source of data. Research in admissions, selection and many other disciplines, consistently 
demonstrates that using multiple sources of reliable data increases decision accuracy. 

Assertion 5: Test scores are biased measures of 
student readiness for minority and underserved 
students 

Each year, ACT releases the Condition of College and Career Readiness report, which 
provides test performance information on the national ACT-tested graduating high school 
cohort. Results are provided for the total group as well as by state and relevant subgroups 
such as underserved minority students, students from low-income families, and first-generation 
students. The findings from these reports highlight the fact that subgroup differences on the 
ACT exist by race/ethnicity and socioeconomic status (SES). For example, Figure 6 shows 
that underserved minorities are much less likely to meet three or more of the ACT College 
Readiness Benchmarks as compared to White and Asian students (ACT, 2015). Unfortunately, 
the general public often misconstrues subgroup differences as evidence of test bias. As stated 
by the Standards for Educational and Psychological Testing (2014), “group differences in 
outcomes do not in themselves indicate that a testing application is biased or unfair” (p. 54). 
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Figure 6. Percent of 2011-2015 ACT-Tested High School Graduates Meeting Three 
of More Benchmarks by Race/Ethnicity. Reproduced from ACT (2015). The Condition 
of College and Career Readiness 2015: National. Iowa City, I A. 

It should also be noted that subgroup differences are not a unique phenomenon isolated to the 
ACT but rather exist across all measures of academic success, including other standardized 
academic measures (e.g., SAT, NAEP) and grades earned in high school and college. 
Subgroup differences are also observed for college enrollment, persistence, and graduation 
rates (Camara & Moore, 2016; Kobrin, Sathy, & Shaw, 2006; Ross, Kena, Rathbun, et al., 
2012). For illustrative purposes, we have provided data on HSGPAforthe 2015 ACT-tested 
graduating cohort to illuminate this point (Camara & Moore, 2016). Though the subgroup 
differences are not as large as what we see for ACT scores, the differences are still quite 
pronounced. For example, as shown in Figure 7, 43% of Asian students and 32% of White 
students obtained a HSGPAof 3.75 or higher whereas only 9% of African American and 17% 
of Hispanic students reached the same level of academic performance. 
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Percentage of 2015 ACT-Tested Students 
by HSGPA and Race/Ethnicity 



■0.00-2.00 ■ 2.00-2.49 ■2.50-2.99 ■ 3.00-3.49 r 3.50-3.74 *3.75+ 


Figure 7. Distribution of HSGPA by Race/Ethnicity. Reproduced from Camara, W. 

& Moore, J (2016). Research you can apply to practice” “Really”! Presented at the 
annual Enrollment Planners Conference, Chicago, IL. 

The distribution of HSGPA values by household income and by highest parental educational 
level are presented in Figures 8 and 9. A similar pattern emerges: students from more affluent 
backgrounds tend to earn higher grades in high school. For example, only 15% of students 
reporting a household income of less than $36,000 earned a HSGPA of 3.75 or higher as 
compare to 36% of students reporting over $80,000. Likewise, only 12% of students who 
reported that neither of their parents finished high school earned a HSGPA of 3.75 or higher 
as compared to 41% for students who reported that one or more of their parents earned a 
graduate degree. 
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Figure 8. Distribution of HSGPA by Household Income. Reproduced from 
Camara, W. & Moore, J (2016). Research you can apply to practice “Really”! 
Presented at the annual Enrollment Planners Conference, Chicago, IL. 
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Figure 9. Distribution of HSGPA by Highest Parental Education Level. Reproduced 
from Camara, W. & Moore, J (2016). Research you can apply to practice. “Really”! 
Presented at the annual Enrollment Planners Conference, Chicago, IL. 
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Research interested in understanding why these subgroup differences exist has found 
that the majority of existing performance gaps can be attributable to differences in course 
taking patterns in high school, high school grades earned, school characteristics, and other 
noncognitive student characteristics (McNeish, Radunzel, & Sanchez, 2015). As displayed 
in Figure 10, students who reported a household income of greater than $80,000 earned 
ACT scores that were noticeably higher than students who reported a household income 
of less than $36,000. The differences ranged from a low of 3.7 points for ACT Science to a 
high of 5.3 points for ACT English. However, after accounting for other student and school 
characteristics, these performance gaps were reduced dramatically with differences ranging 
from only 0.2 to 0.6. Therefore, rather than blaming the test and disregarding the information 
gleaned through the assessment of these academic skills, students would be better served if 
we focused on understanding the social and educational issues that are leaving less affluent 
students ill-prepared for college and the workforce. 



Figure 10. Unadjusted and adjusted mean differences in ACT scores by family 
income. Reproduced from McNeish, D., Radunzel, J. & Sanchez, E. (2015). A 
Multidimensional Perspective of College Readiness: Relating Student and School 
Characteristics to Performance on the ACT. (ACT Research Report No. 2015-6). 
Iowa City, IA: ACT, Inc., 2015. 
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Even though subgroup differences in themselves do not constitute test bias, additional 
analyses evaluating whether subgroup differences represent true differences in academic 
preparation versus construct irrelevant variance should be conducted. For example, whether 
tests results in differential prediction is commonly examined. Differential prediction analyses 
test whether the relationship between test scores and an educational outcome is the same 
across subgroups of interest. For example, if the ACT was a biased measure of underserved 
minority students’ academic preparation and thus an underestimate of their potential, one 
would expect that underserved minority students would perform better in college than 
what their scores would predict. However, research has consistently found that test scores 
over-predict their college performance. That is, underserved minority students tend to earn 
lower grades in college than what one would predict based on their ACT scores. As shown 
in Figure 11, among students with the same ACT Composite score, African American and 
Hispanic students are less likely to earn a FYGPA of 2.5 or higher (and 3.0 or higher) than 
White students (Sanchez, 2013). Using a common regression line based on the total group 
(black line), we see that FYGPA is underpredicted for White students and over-predicted for 
Hispanic and African American students. It is also important to note that HSGPA over-predicts 
FYGPA to a greater extent for underserved minority students than ACT scores; therefore, 
test-optional policies may lead to the admittance of applicants who are not ready for the 
rigorous demands of college level work. 



2.5 or higher (White) 2.5 or higher (Hispanic) 

"3.0 or higher (White) 3.0 or higher (Hispanic) 

2.5 or higher (African American) 2.5 or higher (Total) 

3.0 or higher (African American) — 3.0 or higher (Total) 

Figure 11. Over-prediction of Underserved Students’ Probability of Earning FYGPA 
Levels. Reproduced from Sanchez, E. (2013). Differential Effects of Using ACT® 
College Readiness Assessment Scores and High School GPAto Predict First- Year 
College GPA among Racial/Ethnic, Gender, and Income Groups. (ACT Research 
Report No. 2013-4). Iowa City, IA: ACT, Inc. 
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As shown in Figure 12, the over-prediction of minority performance is not limited to first-year 
outcomes but occurs for long-term outcomes as well, such as degree completion (Radunzel 
& Noble, 2013). We see that among students with the same ACT Composite score, minority 
students are less likely to earn a bachelor’s degree within six years than White students. For 
example, White students with an ACT Composite score of 18 have nearly a 0.40 probability 
of earning a bachelor’s degree within six years as compared to roughly a 0.30 probability for 
minority students. Rather than constituting an underestimate of underserved minority students’ 
potential, ACT scores slightly advantage these students as compared to White students. 



ACTC score 

— White students — Minority students ^—Total group 

Figure 12. Over-prediction of Underserved Students’ Probability of Earning a 
Bachelor’s Degree. Reproduced from Radunzel, J., & Noble. J. (2013). Differential 
Effects on Student Demographic Groups of Using ACT® College Readiness 
Assessment Composite Score, ACT Benchmarks, and High School Grade Point 
Average for Predicting Long-Term College Success through Degree Completion. 
(ACT Research Report No. 2013-5). Iowa City, IA: ACT, Inc. 
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Conclusion 

Scores from the ACT and SAT tests are available for the vast majority of graduates of US high 
schools, ready to be used by colleges and universities for admissions, placement, and to 
identify students who may need extra academic support. For example, approximately 59% of 
high school graduates of 2015 took the ACT test. In certain states, the percentage is much 
higher. Fifteen states adopted statewide ACT testing and exceeded 90% participation rates. An 
additional four states similarly used the SAT statewide in 201 5. With ACT statewide adoption, 
ACT participation rates increase most for male, African American, American Indian, and 
Hispanic students, as well as students with lower family income and students whose parents 
did not attend college (Allen, 2015). Adoption of the ACT or SAT has increased dramatically in 
the past few years as states seek to determine college readiness, reduce duplicative testing, 
and provide colleges with access to a more diverse student cohort. Adoption of the ACT or 
SAT can also be seen as a way to align K-12 with postsecondary expectations, supporting 
articulation of what students need to know and be able to do across educational levels. Test- 
optional policies essentially advocate for ignoring information that is already available and 
used by state assessment systems. 

ACT and SAT test scores also play an important role in regulating high school course-taking 
and grading practices. If HSGPA was the sole admissions criteria used by all colleges and 
universities, students may be less compelled to take the challenging high school courses 
that prepare them for college because those same courses are likely to have higher grading 
standards, resulting in lower HSGPA. Instead, students would be more likely to take the 
courses required for high school graduation and choose easy electives. Combatting this threat, 
grades in college prep courses and strength of high school curriculum are typically used for 
admissions (Clinedinst, 2015). Absent admissions tests, grades in high school courses may 
become even more inflated than they are presently. Schools could easily face pressure — 
intentional or not — from students and the communities they serve to ease grading practices so 
that more of their students would be admitted to colleges of their choice. ACT and SAT scores 
mitigate this problem, or can reveal it when it surfaces. 

A similar checks-and-balances argument supports the use of high school grades and high 
school coursework instead of a test-only admissions model. High school grade and coursework 
data can be examined to ensure that ACT and SAT scores increase for students with higher 
exposure to, and higher grades in, college preparatory courses. 
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When more is more: A Holistic Model of Student 
Success 

Rather than eliminating sources of information, we advocate for using additional sources 
of information in conjunction with HSGPA and test scores to better understand students’ 
academic strengths and weaknesses and their likelihood of success (Camara et al., 2015; 
Mattern et al., 2014). Empirical findings based on ACT-collected data clearly illustrate the 
multidimensional nature of college success and point to the importance of additional factors 
such as academic behaviors and interest-major fit in addition to academic preparation (Allen & 
Robbins, 2008; 2010; Camara et al., 2015; Mattern et al., 2014; Moore, Casillas, & Way, 2015). 
Research shows that higher levels of academic behaviors (as measured by ACT Engage) are 
associated with higher postsecondary degree completion rates within four years of entering 
college, even after accounting for ACT performance (Figure 13; Moore, Casillas, & Way, 2015). 
Specifically, among students meeting three or four Benchmarks, those with high ACT Engage 
College scores attained a timely postsecondary degree at nearly twice the rate as those with 
low ACT Engage College scores (46% vs. 25%). 
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Figure 13. College completion rates by Number of ACT Benchmarks Met and 
Engage Scores. Reproduced from Moore, J., Casillas, A., & Way, J. (2015). Beyond 
Academics: The Role of Behaviors on College Completion. (Inside ACT Research). 
Iowa City, IA: ACT, Inc., 2015 

In sum, taking additional factors into account provides a richer, more holistic view of one’s 
preparedness for future educational success, thereby helping both students and institutions 
make more informed decisions. Whether a student is deciding which schools to apply to or 
what major to declare, or an institution is deciding which students to admit or which students 
are in need of additional support, we contend that more information is better than less. 
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